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ABSTRACT 



The Third International Mathematics and Science Study 
(TIMSS) is the largest and most ambitious international study of mathematics 
and science achievement ever undertaken with more than 500,000 students in 41 
countries being tested in mathematics and science at five different year 
levels. South Africa is the first country in Africa to have participated in 
and successfully completed such a comprehensive international survey in 
science and mathematics education. This report provides detailed information 
about TIMSS and highlights the results of the mathematics and science 
literacy testing of final-year school students in South Africa. The results 
of those students who participated in TIMSS in their final year of schooling 
and student background are discussed in detail. (Contains 15 references.) 
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PREFACE 



In most countries of the world, education and training are important national issues. Cross- 
nationally there is a continuous interest in and effort to improve educational quality. This 
endeavour to improve is directly related to the view that there is a positive relationship between 
the quality of the education and training provided in a country and the ability of that country to 
perform economically in the global context. 

There is also a growing consensus that achievement in mathematics and science subjects is 
symptomatic of the quality of education and training in a country and a good yardstick of a 
country's potential to be economically competitive. The lEA's mathematics and science surveys 
have over the years created an instrument which can be used by participating countries to 
measure where they are and what they need to do in order to improve the quality of their 
mathematics and science education. 

South Africa joined the IEA in 1993 and opted to participate in the Third International 
Mathematics and Science Survey (TIMSS). Even though TIMSS at that stage was already well 
under way, participation was considered to be of such importance that additional steps were 
taken to catch up in order to share in the results and benefits of the project. This report discusses 
the results of the Population 3 survey group, in other words, the final year school students, and 
therefore represents the last group of respondents to the TIMSS questionnaires and achievement 
tests. 

The importance of the TIMSS results is that they provide South Africa with a picture of how 
students achieve in mathematics and science education, and some information about 
environmental issues which may impact on this achievement. The TIMSS results can be used 
to evaluate South Africa's position relative to that of other countries and whatever the 
shortcomings may be of such a comparison, it does provide a rough estimate of the level of 
achievement and the possible shortcomings of the educational system. Deeper analysis of the 
data could also assist in identifying some of the most important factors contributing to the level of 
achievement. 

In the past, in many of the participating countries, the results of IEA investigations lead to 
renewed policy analysis and development. In South Africa the challenge is to respond effectively 
to the TIMSS results. Factors contributing to the poor results have to be carefully analysed and as 
a matter of urgency, remedial actions need to be taken. Given the fact that the poor results are 
probably due to a multitude of factors which financially and logistically cannot be tackled 
simultaneously, care will have to be taken in deciding what to concentrate on first. It would also 
be unwise to set unrealistic objectives. We can only hope to make moderate improvements in the 
next five to ten years. The important point though, is to determine what is viable to achieve and 
then to go all out for the set objectives. 

S.W.H. ENGELBRECHT 
ACTING EXECUTIVE DIRECTOR 
GROUP EDUCATION 
HSRC 
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Executive Summary 

As the twenty-first century approaches, the demand for mathematical, scientific and 
technological understanding and expertise is greater than ever before. Consequently, 
countries around the world have been looking for methods of making teaching and learning in 
these areas more effective in their different school education systems. International studies are 
a means of providing information on student achievement and factors that may play a role in 
such achievement. 

The Third International Mathematics and Science. Study (TIMSS) is the largest and most 
ambitious international study of mathematics and science achievement ever undertaken, with 
more than 500 000 school students in 41 countries being tested in mathematics and science at 
five different year levels (equivalent to Grades 4, 5, 7, 8 and 1 2 in South Africa). This involved 
15 000 schools around the world and 31 different languages. Quality control procedures 
were developed to ensure that the tests written by the students were of a suitable quality to 
ensure the credibility and validity of the study. Detailed sampling procedures were devised for 
selecting schools and students who would participate in the TIMSS study. These procedures 
were designed to ensure that a representative sample was selected from each country. The 
way in which the procedures were implemented was monitored. The International Association 
for the Evaluation of Educational Achievement (IEA), based in the Netherlands, was 
responsible for undertaking TIMSS and other earlier international comparative studies. 

Major research questions guiding TIMSS 

The development of TIMSS was guided by four research questions: 

(1) What are students expected to learn? 

(2) Who provides the instruction? 

(3) How is the instruction organised? 

(4) What have students learned? 

Students participating in TIMSS 

The TIMSS study investigated mathematics and science achievement at different stages of 
schooling and defined the following three target populations: 

Population 1 : the two adjacent grades/standards containing the largest proportion of nine- 
year-old students at the time of testing. 

Population 2 : the two adjacent grades containing the largest proportion of 13-year-old 
students at the time of testing. 

Population 3 : the final year of secondary schooling. 

In the context of the South African study, Population 1 was not included owing to the 
underlying problems with language and the medium of instruction. Population 2 and Population 
3 were tested. Grade 7 and Grade 8 classes were tested together as one group (as 
Population 2) and the Grade 12 classes were tested as Population 3. The Human Sciences 
Research Council conducted the survey in South Africa. 
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This report is the first of a series that will provide information on the performance of the Grade 
12 students in TIMSS in South Africa. 

Key aspects of TIMSS 

Achievement tests are of primary importance to the TIMSS study. These tests were developed 
collaboratively by the countries participating in the study and were subjected to extensive pilot 
studies and field trials. The questions were also reviewed by experts both in assessment and in 
science and mathematics curricula. About a third of the questions required the students to write 
their own answers, rather than select answers from multiple-choice options. 

Comparative studies in education gain more meaning when considered in relation to the 
educational context in which they are done. In TIMSS, data on a considerable number of 
contextual factors included in various questionnaires were collected from principals, teachers, 
students, Education Department officials and curriculum experts. 

Data collected in South Africa 

South Africa is the first country in Africa to have participated in and successfully completed 
such a comprehensive international survey in science and mathematics education. In South 
Africa the decision was made to test Grade 7, 8 and 12 students only, and not to test 
population 1 (grade 4 and 5 pupils) owing to financial constraints and the issue of the medium 
of instruction in the primary schools for this age group. A randomly selected national sample of 
schools, representative of all provinces, race groups, urban and rural communities, was 
identified. 

During 1 995, grade 1 2 students were tested in about 1 50 classes and wrote a general paper 
containing mathematics and science questions. Data from 2 757 of these students were 
included for analysis in the study. The students also completed the same background 
questionnaires. Different questionnaires, which contained questions about socio-economic and 
demographic conditions that could have an impact on the teaching and learning of 
mathematics and science, were administered to school principals. 

In addition to the achievement tests, performance assessment and questionnaires, detailed 
data on curriculum matters were collected. The curriculum was considered, as in previous IEA 
studies, on a number of levels: 

♦ The intended curriculum, which was the curriculum as specified at national/system level 

♦ The implemented curriculum, which was the curriculum as interpreted and delivered by the 
classroom teachers 

♦ The attained curriculum, which was that part of the curriculum learnt by the students, as 
demonstrated by their achievements and attitudes. 

Twenty-two countries, including South Africa took part in the study of final-year students in 
TIMSS. A number of factors were revealed when these countries were compared: 

♦ There was a great variety in the nature of the upper-secondary education systems of the 
countries participating in Population 3. 
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♦ South Africa's TIMSS coverage index 1 was one of the lowest. 

♦ South Africa had the lowest proportion of people living in urban areas. 

♦ South Africa had the second highest public expenditure (as a percentage of the gross 
national product) on education. 

These systemic and demographic factors must be considered when comparing countries and 
their education systems, particularly when performance results are interpreted. 

Highlights of results for Grade 12 

These highlights reflect the results of the achievement tests taken from the international report 
on TIMSS (Mullis, etal. 1998), the HSRC database and some preliminary findings drawn from 
the questionnaires. In-depth analyses and detailed findings regarding the Grade 1 2 students' 
results will be released later. 

Achievement tests 

♦ The overall scores of South African students were significantly lower than those of students 
in other countries, suggesting that South African final-year students have a low level of 
general numeracy and scientific understanding and skills. This was ascertained by 
students' responses to test items focusing on problems applied to the real world. 

♦ South Africa's results did not display any areas in science and mathematics in which 
students performed well. In many of the individual test items, South Africa performed the 
worst. South African students appear to have difficulty with problems that involve graphic 
interpretation. The more difficult multiple-choice questions, as well as free- response items, 
were handled poorly. It would appear that South Africa students generally struggle with 
constructing their own answers, since they performed very poorly on items when 
responses had to be creatively generated. 

♦ There was evidence of language difficulties among the students in the international study, 
including the South African students. The majority of the South African students wrote the 
TIMSS tests in a language that was not their mother tongue. Since the literacy test paper 
comprised largely word problems, it is possible that the language factor had a negative 
impact on achievement. 

♦ South African students with English and Afrikaans as a home language performed 
significantly better than those students with other home languages. 

♦ South Africa was the only country with no significant difference in the performance of boys 
and girls in either mathematics or science. In numerous other countries, boys performed 
better than girls. 

♦ Free State, Gauteng and KwaZulu-Natal performed consistently better than the other 
provinces. Students in these provinces performed above the national average for 
mathematics and science. 



1 



The TIMSS coverage index was calculated for each country and was "an estimate of the percentage of the 
school-learning age cohort covered by theTIMSS final year students sample" (Mullis, el al. 1998:1 7). 
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♦ South African students performed marginally better in mathematics than they did in 
science. 

Preliminary findings from the student questionnaires 

An initial examination by the HSRC of the student questionnaires revealed the following: 

♦ A large proportion of South African students wrote the literacy test in their second or third 
language. 

♦ The average age for South African Population 3 students was 20.1 years. Most of the 
sampled students in other countries were 17-19 years. 

♦ A large majority (85 %) of students in SA were enrolled for biology. About 70 % 2 had 
included mathematics in their curriculum. 

♦ Of the South African sample, 95 % had intentions of studying further, and of these, 75 % 
planned to go to university. The surprisingly high proportion wanting to go to university is 
probably related to the heavy emphasis on academic streams at school level in this 
country, and the perception that university degrees are prestigious and will enhance 
career advancement. The most popular area for further study for South African students 
was engineering, followed by business. That the highest proportion of students elected 
engineering as their intended area of study was surprising. 

♦ South African students in general prefer school biology to physics and chemistry, and this 
seems to support their future study plans which place physics and chemistry at the bottom 
of the list. 

♦ The greatest proportion of students in South Africa reported that the highest educational 
level attained by either parent was primary school, and then to a lesser extent, secondary 
school. 

♦ South African students reported that their homes contained only a limited amount of 
books. Almost 60 % of students had fewer than 26 books in the home, whereas all the 
other countries had only 5-23 % of students reporting this few books at home. 

♦ In South Africa 92 % of students indicated that they did not have a computer at home. 
Concerning the frequency of computer use at school, home or anywhere else, 81 % of 
South African students said that they rarely or never used a computer. In contrast, the 
highest proportion of students in countries such as Australia, Canada, the Netherlands, 
New Zealand, Switzerland and the USA reported that they used computers daily. 3 In the 
information-rich global society in which we live it appears that our final-year students have 
very limited exposure to computer technology in comparison to a number of other 
countries. 



2 There may be overlap in the biology and mathematics groups 

3 Students had to indicate whether they used computers rarely or never, daily, weekly or monthly. 
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♦ South African students reported spending 5 hours a day on homework - the most time 
compared to all other countries. Despite spending so much out of school time on academic 
work, this did not enhance their performance. Students in other countries spent 
considerably less time on homework and the highest achievement was associated with 
1-3 hours of homework a day. 

Many of these findings will be analysed in greater detail and these analyses will be published 
in forthcoming reports. This baseline data has been collected at a time when South Africa is 
undergoing an exciting and challenging reform process in education. The timing of the TIMSS 
study in South Africa is significant and will provide much-needed information to meet the 
educational challenges that will face this country at the beginning of the twenty-first century. 
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Outline of the report 



♦ Chapter 1 offers a description of the design and 
administration of TIMSS. 

♦ Chapter 2 provides an overview of the 
mathematics and science literacy of the study. 

♦ Chapter 3 discusses the findings of the students' 
contextual questionnaire. 

♦ Chapter 4 describes the students' overall 
performance in the mathematics and science 
literacy test. 

♦ Chapter 5 describes the students' performance in 
mathematics and science literacy individually. 

♦ Chapter 6 discusses the students' performance on 
different mathematics and science literacy 
example items. 

♦ The conclusion summarises the preliminary 
analysis of TIMSS. 
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Chapter 1 

Introduction to TIMSS 

Key Points 

♦ TIMSS is the largest and most ambitious international 
study of mathematics and science achievement at 
school level ever undertaken. 

♦ TIMSS was developed to assess the national 
curricula, the school and social environment, and 
achievement in mathematics and science of different 
systems of education. 

♦ Three levels of curriculum were investigated: the 
intended, the implemented and the attained 
curriculum. 

♦ Four main research questions were posed: 

(1) What are students expected to learn? 

(2) Who provides the instruction? 

(3) How is the instruction organised? 

(4) What have students learned? 

♦ Achievement tests and questionnaires were 

administered. Sampling was strictly controlled 
internationally, was drawn randomly and was 
nationally representative 
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Chapter 1 

Introduction to TIMSS 



1.1 Background 

In 1995 the Human Sciences Research Council (HSRC) conducted a survey on mathematics 
and science among 15 000 South African students from more than 400 primary and 
secondary schools, as part of the Third International Mathematics and Science Study known 
as TIMSS. Worldwide more than half a million students in 41 countries participated in the 
project. 

This report represents a summary of the international report on TIMSS, focusing on South 
Africa's participation and, in particular, the study of final-year students is discussed from a 
national and an international perspective. South Africa's results from another part of the study 
(results of the study on Grade 7 and Grade 8) can be found in the report entitled Mathematics 
and Science Performance in the Middle School Years in South Africa - a summary report on the 
performance of South African students in the Third International Mathematics and Science Study 
(Howie, 1997). More information on the South African results can also be found in the 
international reports entitled Mathematics achievement in the Middle School years: lEA's Third 
International Mathematics and Science Study and Science achievement in the Middle School 
years: lEA's Third International Mathematics and Science Study (Beaton et al, 1 996aj and Science 
achievement in the Middle School years: lEA's Third International Mathematics and Science Study 
and Science achievement in the Middle School years: lEA's Third International Mathematics and 
Science Study (Beaton et al., 1996b). 

This report will be followed by others that will investigate issues highlighted here and provide a 
deeper analysis of TIMSS in South Africa as a whole. 

Concern has been growing around the world since the 1960s that investments in education 
need to be related to the outcomes of education, which, in turn, are seen as being able to 
make a substantial contribution to a country's economic prosperity and general well-being. As 
the twenty-first century approaches, the demand for mathematical, scientific and technolo- 
gical understanding and expertise will be greater than ever before. Students at the forefront of 
developments in the future will require very high levels of mathematical and scientific skills. 
These students will need to develop critical thinking, processing and interpreting skills far 
beyond those required a decade previously. As students leave school and enter higher 
education and the workplace, the above skills, in addition to competence in mathematics and 
science, will be crucial. 

With the need for populations to be better educated (in South Africa's case, the basic need for 
literacy and numeracy) in the climate of shrinking national budgets, countries around the world 
have been looking for methods of making teaching and learning more effective. International 
studies are a means of providing information on student achievement and the factors that play 
a role in such achievement. They "help to improve an understanding of educational systems. 
As there are no absolute standards in educational achievement, comparative studies are 
essential to provide policy makers and educators with information about the range of 
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educational quality in relation to other national systems. Studies therefore contribute to setting 
realistic goals for education systems, as well as monitoring school quality” (Plomp, 
1992:278). Secondly, comparative studies may also be helpful in "understanding the causes 
of observed differences in student performance, by exploring cross-nationally relations 
between school achievement and such factors as curricula, amount of time spent on school 
work, and teacher training, parental support and many other explanatory measures” (Plomp, 
1 992:278). The challenge will always be to learn more about effective teaching and learning, 
both for educators and for policy makers in the education field. 

In South Africa, poor matriculation results continue to dominate the news at the end of every 
school year. During 1995 and 1996 the best school attendance figures were witnessed in 
years, but poor results were still evident. The story was repeated in 1 997 when less than half 
of the country's matriculants passed, despite changes and reforms in the Department of 
Education. The reasons for the failure are varied, and any reforms that are implemented need 
time to make an impact. A system is required whereby the impact of these changes can be 
monitored. This concept of monitoring has long been recognised internationally and has been 
adopted by other countries introducing reform into their education systems. 

TIMSS is the largest and most ambitious international study of mathematics and science 
achievement at school level ever undertaken. It is the first time that mathematics and science 
studies have been combined as an integrated study. TIMSS is also the largest comparative 
study of its kind conducted under the auspices of the International Association for the 
Evaluation of Educational Achievement (IEA), which is based in the Netherlands. The IEA is an 
independent international grouping of national research institutions and governmental research 
agencies. Its primary purpose is to conduct large-scale comparative studies of educational 
achievement, with the aim of gaining a deep understanding of the effects of policies and 
practices within and across systems of education. The IEA has conducted more than 1 5 studies 
of achievement involving groups of different countries since its inception in 1959. 

The IEA officially launched TIMSS in 1994. The project was undertaken in more than 60 
countries across the world. Highly developed countries and developing countries, from both 
the northern and the southern hemispheres, were included. Of the 63 countries that started the 
study, only 41 completed it. South Africa was the only country in Africa to do so. 

TIMSS was developed to assess the national curricula, the school and social environment and 
achievements in science and mathematics in the participating countries. TIMSS tests were 
designed to measure mathematics and science achievement in order to help inform 
governments, policy makers and educators about the mathematics and science proficiency 
of their students at key points in the educational process. The questionnaires were aimed at 
collecting information about factors related to students' learning of mathematics and science. 

The first part of this report gives an overview of the entire project of TIMSS, including its 
conceptual framework, various components and the design of the achievement instruments. 
The second part of the report summarises the science and mathematics achievement of the 
South African Grade 12 students and sketches the students' background information. 

1 .2 Conceptual framework for TIMSS 

IEA studies traditionally have recognised the importance of the curriculum as a variable for 
explaining differences among national school systems and accounting for student outcomes. 
These studies represented an effort to understand education systems and to make valid 
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comparisons between them. The curricula and teaching practices of different national systems 
were investigated and compared with the student outcomes. These three factors became the 
focus areas forTIMSS. It was believed that differences in achievement could be explained in 
terms of variations in curriculum, teaching practices and other variables. It was also hoped that 
the study would help countries to evaluate national curricula and provide a research basis for 
future national curriculum reform. 

The conceptual model for TIMSS was derived mainly from the models used in earlier IEA 
studies, especially for SIMS (Second International Mathematics Study) and SISS (Second 
International Science Study). In this model three "levels" of curriculum are envisaged (see 
Figure 1.1): the intended, the implemented and the attained curriculum. The educational 
environment, made up of a variety of factors, should be understood from the perspective of 
these three curriculum levels. It is believed that there are factors outside formal schooling that 
affect the student's achievement. There is a unique set of contextual factors that influence the 
educational decisions for each level. of the curriculum (Martin and Kelly, 1996). 



Figure 1:1 TIMSS conceptual framework* 
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Robitaille's model, adopted by TIMSS as its conceptual framework, provides a rationale and 
context for the key research questions in TIMSS. Four questions are central to the study: 

(1) What concepts, processes and attitudes regarding mathematics and science have 
students learnt and what factors are related to their opportunity to learn these concepts, 
processes and attitudes? 
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(2) How do education systems differ in the intended learning goals for mathematics and 
science, and what characteristics of education systems, schools and students are related 
to the development of these learning goals? 

(3) What opportunities are provided for students to learn mathematics and science, how do 
instructional practices in mathematics and science vary among education systems, and 
what factors are related to this variation? 

(4) How are the intended, implemented and attained curricula related with respect to the 
context of education, the arrangements for teaching and learning and the outcomes of 
the education process? 

1.3 TIMSS curriculum framework 

The concept of the curriculum as a variable requires a framework for the description of the 
three levels of the curriculum. In the context of the TIMSS curriculum framework (Robitaille et 
al., 1993), the curriculum consists of the concepts, processes and attitudes of school 
mathematics and science, that are intended for, implemented in, and attained during students' 
schooling experiences. This framework allows for a given assessment question (item) or 
learning material to be categorised in detail. The items can be categorised in terms of subject 
matter (e.g. proportionality), performance expectations (e.g. problem-solving) and perspec- 
tives (e.g. positive attitudes towards mathematics, science and technology). The detailed 
categories within the science and mathematics frameworks differ, but the structure and 
rationale of the two are the same, allowing for comparisons across the two curriculum areas 
(see Figure 1.2 and Figure 1.3.). 
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Figure 1 .2: Three aspects and major categories of the mathematics framework 



Content aspect 

♦ Numbers 

♦ Measuremenr 

♦ Geometry: position ... 

♦ Geometry: symmetry ... 

♦ Proportionality 

♦ Functions, relations, equations 

♦ Data, probability, statistics 

♦ Elementary analysis 

♦ Validation and structure 

♦ Other content 



Performance expectations aspect 

♦ Knowing 

♦ Using rourine procedures 

♦ Investigating and problem 
solving 

♦ Mathemaricai reasoning 

♦ Proportionality 

♦ Communicatinc 



Perspective aspect 

♦ Attitudes 

♦ Careers 

♦ Participation 

♦ increasing interesr 
♦. Habits of mind 



(Robitaille et al. 1993:46) 
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Figure 1 :3 Three aspects and major categories of the science framework 



Content aspect 

♦ Earth sciences 
^ Life sciences 

♦ Physical sciences 

♦ Science, technology, 
mathematics 
History of science and 
technology 

♦ Environmental issues 
& Nature of science 

Science and other disciplines 






(Robitailie ef 



Performance expectations aspect 

♦ Understanding 

^ Theorizing, analyzing, solving 
problems 

♦ Using tools, routine procedures 

♦ Investigating the natural world 

♦ Communicating 



I ^ 



Perspectives aspect 
Attitudes 
^ Careers 
^ Participation 

♦ Increasing interest 

♦ Safety 
+ Habits 
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The content aspect represents the content of school mathematics or science, depending on the 
framework selected. The performance expectations aspect is the cognitive (thinking) 
behaviour dimension in earlier IEA studies, which has been reconceptualised. The aim of 
this particular aspect is to describe the many kinds of performances or behaviours that one 
would expect from students. The perspectives are attitudes that students develop by exposure- 
to a particular curriculum. 

Each of the three aspects (content, performance expectations and perspectives) is divided into 
a number of categories and subcategories. The TIMSS framework can be described as a multi- 
aspect, multi-category system, where a test question can be related to any number of 
categories within each aspect or aspects. An achievement question must therefore be 
associated with numerous combinations of aspect categories in the TIMSS framework. 

It is accordingly believed that this framework is well suited to deal with the complexity of 
student activities emerging from various national reforms in mathematics and science. It is also 
suited to the complex, integrated performances expected from students in the new forms of 
assessment emerging with the curricular reforms. 

1 .4 Students tested in TIMSS 

TIMSS identified and selected three focus areas in the school education system. In the 
international context these were referred to as Population 1, Population 2 and Population 3. 
Population 1 consisted of students in those grades containing the largest proportion of nine- 
year-old students. Population 2 consisted of students in the two adjacent grades containing the 
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largest proportion of 1 3-year-old students. Population 3 consisted of students in their last year 
of formal schooling. The age of the students was taken as the age of the students when they 
wrote the achievement test (Martin and Kelly, 1996). 

In the context of the South African study, Grades 4 and 5 represented Population 1 . At the 
time of the investigation the mother tongue was the medium of instruction up to Grade 2. This 
population was not included as no resources (human and financial) were available for 
translations into other languages. Grade 7 and Grade 8 represented Population 2 and Grade 
12 represented Population 3. Grades 7, 8 and 12 were all tested in South Africa, allowing 
South Africa to be included in the Population 2 and Population 3 of TIMSS. This report focuses 
on the achievement of this last group of students. 

The students in Grade 1 2, who were in the selected schools, were tested regardless of the 
subjects they were studying. Internationally, those Population 3 students who were 
specialising in mathematics or physics were identified as subgroups and were given 
specialised tests. In South Africa, students do not specialise to this degree and therefore the 
Grade 1 2 students who were tested wrote a general science and mathematics literacy 
achievement test. No specialised tests were written. 

1 .5 General design of TIMSS 

The TIMSS research was conducted at three levels: the curriculum analysis level, the 
achievement tests level and the questionnaires level. Figure 1 .4 illustrates the instruments 
(achievement tests and questionnaires) that were used. 

Four questionnaires were administered at the national (system) level at various times in the 
course of the study. Two of these questionnaires dealt with the organisational structure, 
courses, demographics and teacher credentials, and were completed by people knowledge- 
able about the structure of their education system. Two other questionnaires were for 
curriculum specialists, seeking information on the national level curriculum plans, reforms, issues 
and policies with respect to mathematics and science curricula. Both the questionnaires were 
completed either by a mathematics or by a science curriculum specialist. The "specialist'' 
questionnaires provide important information on present and future curricular goals and content 
changes - information not contained in curriculum guides. Questions were asked about recent 
curriculum innovations, national assessment practices, instructional goals, the availability of 
textbooks and teachers' manuals and policies on curriculum guides, assessment and student 
tracking. The purpose of these questionnaires was to gain an understanding of the contexts of 
the education systems, and their impact on the intended and implemented curricula. Data at 
the national level ‘were supplemented with data collected at school level, in order to provide 
information on how education might vary within a country. 

A school questionnaire was designed for the principal of each sampled school. The results 
from these questionnaires should give a good idea of the kinds of schools in the education 
system. Among the topics addressed in this questionnaire were enrolment, demographics and 
subject selection, as well as administrative, curricular, budgetary and social issues. The 
questionnaires administered to the primary and secondary schools were similar in content, with 
some questions being modified or omitted. 

TIMSS developed three teacher questionnaires to obtain information on the curricula 
implemented at school. These included two questionnaires at Population 2 level, one of 
which was designed for the mathematics teachers and one for the science teachers. The other 



Figure 1 .4: The relationship of TIMSS data collection instruments to the conceptual framework 
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questionnaire was at Population 1 level; Population 3 had no teacher questionnaire. The 
questionnaires included five topics: the teacher's background, attitudes to teaching and 
learning, expectations for students, instructional practices, and opinions on mathematics and 
science education. 

TIMSS also developed three different questionnaires for students, although they were similar in 
organisation and content. There was one for each TIMSS population group tested. Each 
included questions on the student's background, opinions and attitudes to mathematics and 
science. 



1 .6 TIMSS achievement instruments 

Traditionally, large-scale surveys conducted by the IEA and other bodies have used multiple- 
choice questions. Tests using the multiple-choice format are very popular since the test 
conditions can be standardised, the cost is low and they can be machine-scored. However, 
there has recently been a growing awareness among educators that some important 
achievement outcomes are either difficult or impossible to measure using the multiple-choice 
format. Communicating mathematical or scientific findings or constructing a mathematical 
proof requires skills for which the multiple-choice questions appear to be unsuitable. It was 
therefore decided that TIMSS should employ a variety of questions to increase the coverage 
of the desired outcomes of school mathematics and science education. Four different types of 
questions were included in the pool of TIMSS questions: multiple-choice questions, short 
answer questions, extended answer questions and performance tasks. (In TIMSS the short- 
answer and the extended answer questions were referred together as free-response items). As 
in all other countries, these achievement tests were written in the students' language of 
instruction, which for South African students in Grade 7, 8 and 12 was English or Afrikaans. 

The multiple-choice questions consisted of a question and four or five choices of answer, of 
which only one was the correct answer. Students were encouraged to choose the answer 
they thought was the best when they were unsure. 

In both the short answer and the extended response questions, students were required to write 
their responses and these were coded using a set of two-digit codes developed for use in 
TIMSS. The first digit was coded for correctness of the answer and the second digit was a 
code to identify the quality of the students' responses. For the short answer items, the students 
were required to write a brief verbal or numerical answer. The items were coded correct or 
incorrect, and the students were not required to show details of the approach they used or the 
calculations they performed on such questions (Martin and Kelly, 1996:7-7). 

The multiple-choice, short answer questions and extended response questions were randomly 
distributed, in different groups or clusters of questions, throughout the test booklets. 

The performance tasks were designed to assess some of the skills and abilities that could not 
be assessed readily by a written test. Tasks were performed in small groups and these groups 
were observed by the researchers in participating countries. However, South Africa did not 
use this form of assessment owing to financial and other constraints. /> 
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1 .7 Sampling for the TIMSS Population 3 study 

The sampling procedure was designed and strictly controlled by the international study group 
to ensure the statistical validity of the study. A random sample of students, representative of 
the country, was to be selected. The sample of schools that was used for Population 3 (grade 
12) was the same as that used for Population 2. This was largely a financial decision, making 
the fieldwork more time and cost-effective. 

The sample for Population 2 was randomly drawn from all schools that had more than 40 
students in Grade 8 (Std 6) in 1991 . This was done as follows (Claassen, 1995) 

♦ For the Western Cape, schools were arranged from the smallest to the largest, according 
to the number of students in Grade 8. 

♦ Immediately below the list for the Western Cape, a similar list for the Northern Cape 
schools was compiled, but this time arranged from the largest to the smallest in terms 
of Grade 8 student numbers. 

♦ This process was continued for the other seven provinces. 

♦ An appropriate interval was determined so that 150 schools would be chosen from 
the list. 

♦ A random number was generated to select the first school, after which the remaining 
149 followed automatically. 

♦ In each of the selected schools (as determined above), one of the Grade 1 2 classes 
was randomly selected and the tests administered to that class only. 

1 .8 Administration of Population 3 in South Africa 

The achievement test was administered between August and December 1995. The scoring of 
the free- response questions was completed in January 1 996. The total number of students 
included for analysis are indicated below in Table 1.1. Some of the selected schools were 
unable to participate for various reasons. The number of classes participating in the study 
whose scores were reflected in the international reports, was 90. The number of participating 
students whose scores were included in the international study was 2 757. 4 The scores of the 
pupils were weighted so that the resulting mean score obtained from the study would be an 
unbiased estimate of the population mean. 



4 3 695 students were tested in total, but after exclusions (such as students who were repeating Grade ] 2) the figure 

dropped to 2 757. 



TIMSS-SA 



GRADE 12 



MATHEMATICS AND SCIENCE LITERACY 



Table 1.1: Number of students participating in TIMSS Population 3 per province 5 



Province 


Number of 
Grade 12 


Eastern Cape 


570 


Free State 


204 


Gauteng 


368 


Kwazulu-Natal 


534 


Mpumalanga 


292 


Northern Province 


358 


North West 


397 



5 Twenty-five students were tested in the Western Cape and nine in the Northern Cape. There were problems in gaining 
access to schools in the Western Cape, making the realised sample substantially lower than the intended sample. Due 
to the low density in the Northern Cape province, initially a proportionately smaller sample was drawn. When access 
to a number of schools was denied, the size of this sample was significantly affected. 
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Chapter 2 

Background to Population 3 
results 

Key Points 

♦ There was a great variety in the nature of the upper- 
secondary education systems of the countries 
participating in Population 3. 

♦ South Africa's TIMSS coverage index 6 was one of 
the lowest. 

♦ South Africa had the lowest proportion of people 
living in urban areas. 

♦ South Africa had the second highest public 
expenditure (as a percentage of the gross national 
product) on education. 



6 



The TIMSS coverage index was calculated for each country and was "an estimate of the percentage of the 
school-learning age cohort covered by the TIMSS final-year students sample" (Mullins, I.V.A. el al. 1998: 1 7). 
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Chapter 2 

Background to Population 3 results 



2.1 Introduction 

The concerns on the part of industry and within communities about mathematics and science 
education are increasing. There is a growing need for better qualified workers in the fields of 
science and technology to enter the workplace, and an increased interest in what school- 
leavers know and are able to do in mathematics and science. 

The TIMSS data for the final-year students can be used as a basis for determining the concepts 
and understanding of these students in the fields of mathematics and science. It is also a means 
to establish how effectively these students will use their knowledge and understanding once 
they have left secondary school. 

The TIMSS results and findings concerning this age group may be used in several ways. It is 
hoped that this information will be used constructively to inform policy makers and practitioners 
in order to reconstruct and develop education in South Africa and other countries. Furthermore, 
it may be used to mobilise communities to support the implementation of new policies in the 
fields of mathematics and science in the schools and training institutions, as well as to support 
students wishing to enter these fields. TIMSS also provides a means of comparing education 
systems and policies. This should be more important to policy makers and practitioners than 
simply comparing the final rank order of the countries participating. The similarities and 
differences between different countries' education systems, made apparent by the data 
collected, have served in the past to motivate changes in various governments' policies with 
regard to mathematics and science. 



2.2 Countries participating in TIMSS Population 3 

The countries that took part in the mathematics and science literacy tests were the following: 

♦ Australia 

♦ Austria 

♦ Canada 

♦ Cyprus 

+ The Czech Republic 

♦ Denmark 

♦ France 

♦ Germany 
+ FHungary 
4 Iceland 




25 



28 



TIMSS-SA 



GRADE 12 



MATHEMATICS AND SCIENCE LITERACY 



♦ Israel 

♦ Italy 

♦ Lithuania 

♦ The Netherlands 

♦ New Zealand 

♦ Norway 

♦ The Russian Federation 

♦ Slovenia 

♦ South Africa 

♦ Sweden 

♦ Switzerland 

♦ United States of America 

2.3 Students tested in Population 3 

There was a great variety in the nature of the upper-secondary education systems of the 
countries participating in Population 3. Different types of upper-secondary education were 
offered in different countries and these were in the form of 

♦ schools offering comprehensive education, 

♦ schools offering a single track e.g. vocational training, and 

♦ schools offering different tracks within the school, e.g. academic, vocational, technical. 

Of course there was an additional complication as the definition of academic, vocational and 
technical also differed from country to country. Defining the final-year of schooling across all 
the participating countries was also problematic, as there were variations in the grades 
representing the final year of schooling. 

Therefore, it is important to understand the nature of the different upper-secondary education 
systems in order to make valid comparisons between countries. 

2.4 TIMSS coverage index (TCI) 

One of the important differences between education systems of different countries is the 
proportion of the age group that completes upper-secondary education. Since the 1960s 
there has been a significant increase in the number of students achieving this level. 

The TIMSS coverage index (TCI) was calculated for each country in order to provide 
information about the number of pupils in the school-leaving age group still at school, and that 
represented the TIMSS sample. The TCI is "an estimate of the percentage of the school- 
leaving age cohort covered by the TIMSS final-year students sample" (Mullis etal., 1 998: 1 7). 
"The TCI reflects the exclusion from the sample, of, for instance, students who were omitted 
who had dropped out of school, tracks or educational programmes which were not covered 
by the TIMSS sample. The TCI was calculated by "forming a ratio of the size of the student 
population covered by the TIMSS sample, as estimated from the sample itself, and the size of 
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the schooling-leaving age cohort, which was derived from official population census figures of 
each country "(Mullis, et oi, 1998:17). 

Therefore, countries with high TCIs have most of their students of school-leaving age still at 
school and have covered this population with their TIMSS sample. On the other hand, 
countries with low TCIs have fewer students in school or have excluded some components of 
their system from their sample. 

In the table below, the TCIs for each country are given. The section of the school-leaving 
group not covered by the sample is presented in two groups, these being the systems 
components excluded and secondly these students electing not to complete the upper- 
secondary education. As can be seen from Table 2. 1 , South Africa has a TCI of just 48,9 %, 
indicating that the TIMSS sample of final-year students covered just less than half the school- 
leaving age group. 



2.5 Characteristics of countries participating in TIMSS 
Population 3 

The results of comparative studies should also be seen in the broader context in which students 
are educated and the factors that play a role. Some of the demographic factors are presented 
for each country in Table 2.2. This table includes information on the size and density of the 
population, the size of the country, the proportion of urban dwellers. South Africa is the least 
urbanised of the countries tested and has the lowest percentage of students in secondary 
school. 

Table 2.3 presents information concerning public expenditure on education. It includes 
information on the gross national product (GNP) per capita and public expenditure as a 
percentage of the GNP. As can be seen, South Africa's GNP is the lowest among the 
countries participating in TIMSS. However, the percentage of GNP spent on education in 
South Africa is second only to that of Norway. 
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Table 2.1 : TIMSS coverage index 



Country 


TIMSS 

coverage 

index 

1%) 


Sample 

exclusions 

(%) 


Others 
not covered 
(%) 


Notes on exclusions 


Australia 


68,1 


4,0 


27,9 




Austria 


75,9 


16,8 


7,3 


Colleges lasting less than 3 years 


Canada 


70,3 


19,5 


• 10,2 


Ontario students graduating in 
December excluded 


Cyprus 


47,9 


13,5 


38,6 


Private and vocational school 
excluded 


Czech Republic 


77,6 


0,0 


22,4 




Denmark 


57,7 


1,3 


41,0 




France 


83,9 


0,9 


15,3 




Germany 


- 


- 


- 




Greece 


10,0 


56,8 


33,2 


Only students having taken 
advanced mathematics and physics 
included 


Hungary 


65,3 


0,1 


34,6 




Iceland 


54,5 


0,0 


45,4 




Israel 


- 


- 


- 




Italy 


51,5 


0,5 


48,0 




Latvia 


3,0 


16,8 


80,3 


Only students having taken physics 


Lithuania 


42,5 


0,0 


57,5 




Netherlands 


78,0 


20,6 


1,4 


Short vocational and apprenticeship 
programmes excluded 


New Zealand 


70,5 


0,0 


29,5 




Norway 


84,0 


1,3 


14,8 




Russian Federation 


48,1 


36,3 


15,7 


Vocational schools and non-Russian 
speaking students excluded 


Slovenia 


87,8 


0,0 


12,2 




South Africa 


48,9 


0,0 


51,1 




Sweden 


70,6 


0,0 


29,4 




Switzerland 


81,9 


2,1 


16,0 




United States 


63,1 


0,6 


36,4 





(Mullis, I.V.A. etal., 1998:18) 
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Table 2.2: Demographic characteristics of TIMSS countries 



Country 


Population 

Size 

(1,000) 


Area of 
Country 
(1000 square 
kms) 


Density 
(population 
per square 
km) 


Percentage 
of population 
living in 
urban areas 


Percent in 
Secondary 
School 


Australia 


17843 


7731 


2,29 


84,8 


84 


Austria 


8028 


84 


95,28 


55,5 


107 


Canada 


29248 


9976 


2,9 


76,7 


88 


Cyprus 


726 


9 


77,62 


53,6 


95 


Czech Republic 


10333 


79 


130,99 


65,3 


86 


Denmark 


5 205 


43 


120,42 


85,1 


114 


France 


57928 


552 


104,56 


72,8 


106 


Germany 


81 516 


357 


227,39 


86,3 


101 


Greece 


10426 


132 


78,63 


64,7 


99 


Hungary 


10261 


93 


1 10,03 


64,2 


81 


Iceland 


266 


103 


2,56 


91,4 


103 


Israel 


5383 


21 


252, 1 4 


90,5 


87 


Italy 


57120 


301 


189,36 


66,6 


81 


Latvia 


2547 


65 


40,09 


72,6 


87 


Lithuania 


3 721 


65 


57,21 


71,4 


78 


Netherlands 


15381 


37 


409,30 


88,9 


93 


New Zealand 


3493 


271 


12,78 


85,8 


104 


Norway 


4 337 


324 


13,31 


73,0 


1 16 


Russian Federation 


1 48 350 


17075 


8,7 


73,2 


88 


Slovenia 


1 989 


20 


97,14 


62,7 


85 


South Africa 


40539 


1 221 


_ 32,46 


50,5 


77 


Sweden 


8 781 


450 


19,38 


83,1 


99 


Switzerland 


6994 


41 


168,03 


60,6 


91 


United States 


260650 


9809 


27,56 


76,0 


97 



(Mullis, I.V.A. e/a/., 1998:22) 
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Table 2.3 Public expenditure at primary and secondary levels in TIMSS countries 



Country 


Gross National product 
per capita (US dollars ) 7 


Public expenditure on 
education as % of gross 
National Product 


Australia 


17 890 


3,69 


Austria 


24 950 


4,24 


Canada 


19 570 


4,62 


Cyprus 


10 380 


3,60 


Czech Republic 


3 210 


3,75 


Denmark 


2 81 10 


4,80 


France 


23 470 


3,61 


Germany 


25 580 


2,43 


Greece 


7 710 


2,27 


Hungary 


3 840 


4,31 


Iceland 


24 590 


4,77 


Israel 


1 4410 


3,72 


Italy 


19 270 


2,89 


Latvia 


2 290 


2,85 


Lithuania 


1 350 


2,18 


Netherlands 


21 970 


3,3 


New Zealand 


13 190 


3,15 


Norway 


26 480 


5,26 


Russian Federation 


2 650 


_ 


Slovenia 


7 140 


4,2 


South Africa 


3 010 


5,12 


Sweden 


23 630 


4,92 


Switzerland 


37 180 


3,72 


United States 


25 860 


4,02 



(Mullis, I.V.A. etai, 1998:23) 



7 These figures are estimates for 1 994. 
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Chapter 3 

Student background 

Key Points 

♦ A large proportion of South African students wrote 
the literacy test in their second or third language. 

♦ The average age for South African Population 3 
students was 20. 1 years. 

♦ The majority (85 %) of students in SA were enrolled 
for biology. About 70 % had included mathematics 
in their subject choice. 

♦ Of the South African sample, 95 % had intentions of 
studying further, and of these, 75 % planned to go to 
university. 

♦ The most popular area nominated for further study 
by South African students was engineering. 

♦ Students in general prefer biology to physics and 
chemistry. 

♦ The greatest proportion of students in South Africa 
reported that the highest educational level attained 
by either parent was primary school and then, to a 
lesser extent, secondary school. 

♦ Close to 60 % of students in South Africa had less 
than 26 books in the home. 

♦ In South Africa 92 % of students indicated that they 
did not have a computer at home. 

♦ South African students reported spending five hours 
a day on homework - the most time compared to all 
other countries. 
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Chapter 3 

Student background 



3.1 Introduction 

To help explain the mathematics and science literacy achievement results across countries, 
TIMSS elicited a range of descriptive information from various educational roleplayers. One 
component of sketching the contexts supporting education in different countries was to gather 
information from students themselves concerning their attitudes, perceptions, backgrounds and 
in- and ouf-of-school activities. As these can impact on academic performance it is important 
to understand the conditions that support or limit student achievement. 

This chapter examines students' attitudes, perceptions and background information given in 
response to several questions posed to them. A number of these questions revolved around 
their current training and their plans for future education. In order to examine the impact that 
the students' home and social environments may have on academic achievement, some of the 
questions looked at the issue of educational resources in the home. This is particularly relevant 
to the South African case as disparities in resources have pervaded the South African society 
for many years. In addition, the use and effectiveness of both calculators and computers 
remain debates in mathematics and science education, and a number of questions addressed 
these issues. A further group of questions involved whether or not students typically spend their 
out-of-school time in ways that support their in-school academic performance. 



3.2 Student biographical information 

Of the South African student sample, 51 % were female and 49 % were male. 

It appears that a large proportion of the South African students who wrote the literacy test did 
so in their second or third language. About 19 % indicated that they always or almost always 
spoke the language of instruction at home, whereas approximately 72 % indicated that they 
sometimes spoke the language of learning at home. The remaining 9 % never spoke the 
language of instruction at home. In the South African context language is likely to be a factor 
influencing performance. Further comparative analysis of achievement levels with other 
countries in which students wrote the performance tests in their second or third language may 
uncover some interesting findings. 

When asked how many people lived in their home the highest proportion of South African 
students reported about 5-6 people (about 17% for each). A slightly lower proportion 
indicated that about 4, 7 or 8 people lived in their home. Small households of three people or 
fewer were much less common. It is also interesting that a much higher proportion of students 
lived with their mothers (83 %) than with their fathers (48 %). 8 About 80 % of students 
reported that they were residing at home as opposed to living with relatives away from home. 



8 



Students who had their mothers living at home with them may also have had their fathers at home (the 48 % 
could be included in the 83 % value). A number of other people may also have been living in their house. 
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The average age of students participating in the mathematics and science literacy study is 
shown in Figure 3. 1 . 



Figure 3.1: The average age of population 
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Apart from Iceland (not shown in Figure 3.1), which had an average age of 21,2, the South 
African students appear to be above the average age in comparison to all the other countries. 
Most countries' participating students were roughly 17-19 years old. That South African 
students were among the oldest can be attributed to a number of factors such as late entry into 
school, and repetition of the earlier grades. 



3.3 Educational aspirations 

3.3. 1 Subject preferences 

The approach in many countries is for students in the upper secondary classes either to select 
or to be allocated to educational tracks that fit in with their abilities or interests. As has already 
been alluded to, because of these cross-national differences it is difficult to define international 
categories that are comparable. Four broad categories, however, were distinguishable - 
academic, technical, vocational and general. South Africa was classified as having an 
academic programme. All 22 participating countries offered an academic programme and a 
large majority (1 7) also offered a vocational one. 



9 The countries represented in the graph are Australia, Canada, the Czech Republic, Denmark, Germany, the 
Netherlands, New Zealand, the Russian Federation, South Africa, Sweden and the United States of America. 
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Across a broad spectrum of students participating in the study it is interesting to determine 
whether students elect mathematics and science as part of their curriculum. Table 3. 1 illustrates 
that almost 70 % of students in the South African sample included some type of mathematics 
course during their final year of schooling. Internationally the proportion of students taking 
mathematics varied considerably from 54-100 %. In most countries, including South Africa, 
more males than females reported that they were currently taking mathematics. In South Africa 
67 % of females and 71 % of males indicated that they were enrolled for mathematics. 

Table 3.1 : Proportions of final-year South African students taking mathematics 



Level of mathematics taken 


% 


Did not take mathematics up to std 1 0 


29,7 


Functional grade mathematics std 10 


3,9 


Standard grade mathematics std 10 


39,8 


Higher grade mathematics std 10 


26,5 



Compared to the enrolment- in mathematics, fewer students in the sample were enrolled for 
physical science (more likely than not this covered physics and chemistry). Just over 50 % of 
the South African sample took physical science. The same proportion of males and females 
reported taking science in South Africa. There were fewer countries showing substantial 
differences in the percentages of males and females taking science than there were countries 
showing gender differences in numbers of pupils taking mathematics. 



Table 3.2: Proportions of South African students enrolled for science courses 



Level of science 
course taken 


Most advanced course 
taken in physics (%) 


Most advanced course 
taken in chemistry (%) 


Did not take physical science up to 
std 10 


45,6 


46,7 


Std 10 functional/standard grade 
physical science 


21,7 


21,1 


Std 10 higher grade physical 
science 


32,6 


32,1 



Approximately 85 % of students in the TIMSS sample in South Africa were enrolled for 
biology. This could be explained by the high proportion of schools traditionally offering 
biology as a compulsory subject, the fact that students found the subject appealing or that 
students perceived it to be an easy subject to pass. 

The proportion of students taking geography was similar to that taking physical science. Only 
approximately half of the South African sample was studying geography in their final year of 
school. 
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3.3.2 Future study plans 

A few questions focused on the prospect of further study. Ninety-five percent of the South 
African sample reported that they had intentions of pursuing tertiary studies. Of that cohort, 
75 % planned to go to university. 10 Along with Slovenia this statistic constituted the highest 
percentage of students intending to go to university. The fact that considerably fewer students 
in other countries elected to go to university can be explained by the fact that numerous 
countries have well-developed vocational or technical programmes, and students then plan 
tertiary studies in similar programmes. In general, the students who intended studying at a 
university (as opposed to vocational training) showed higher achievement scores in 
mathematics and science literacy. 

Students were asked to indicate their proposed career directions. The most popular area 
nominated for further study by South African studenfs'(see Table 3.3), was engineering (all the 
conventional engineering domains). The business field, which includes accounting, marketing, 
finance, administration and management, was the second most popular choice. The health 
sector also appeared to be appealing to students, with a similar proportion of students hoping 
to study in non-science- related fields. The fields of chemistry and physics were the least 
attractive to students. Only 1,8 % and 0,9 % of students reported any interest in studying 
further in chemistry and physics respectively. 

Table 3.3: Students' preferences concerning intended future study 



Area of intended study 


% 


Biological sciences (e.g. botany, ecology, zoology) 


10,7 


Business (e.g. accounting, marketing, finance, administration, 
management) 


17,2 


Chemistry (e.g. chemistry, biochemistry) 


1,8 


Computer and Information sciences (e.g. sytems analyst) 


6,5 


Earth sciences (e.g. geology, oceanography) 


3,0 


Engineering (e.g. chemical, civil, electrical, mechanical engineering) 


18,4 


Health occupations (e.g. dental assistant, practical nursing, veterinary 
assistant, X-ray technology) 


11,9 


Health sciences (e.g. dentistry, medical doctor, optometry, pharmacy, 
physical therapy, registered nursing, veterinary medicine) 


13,6 


Mathematics (e.g. calculus, statistics) 


4,6 


Physics (e.g. astronomy, physics) 


0,9 


Other fields of study 


11,5 




1 0 Due to difficulties in arriving at a consistent definition of "university" internationally, cross-national comparisons 
were somewhat inappropriate. 
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3.4 Students' attitudes towards mathematics and science 

3.4. 1 Previous performance in mathematics and science 



Table 3.4: Students' reports on how well they perform in mathematics and science 



Doing well in mathematics 




Doing well 


in science 






Agree or 


Disagree or 


Agree or 


Disagree or 




strongly agree 


strongly disagree 


strongly agree 


strongly disagree 






Mean 




Mean 




Mean 




Mean 




% of 


mathematics 


% of 


mathematics 


% of 


science 


% of 


science 


Country 


students 


literacy 


students 


literacy 


students 


literacy 


students 


literacy 






score 




score 




score 




score 


Australia 


72 


544 


28 


477 


73 


554 


27 


470 


Canada 


67 


542 


33 


476 


75 


548 


25 


489 


Czech 


55 


487 


45 


441 


71 


500 


29 


463 


Republic 


















Denmark 


76 


566 


24 


498 


72 


535 


28 


469 


Germany 1 1 


















Netherland 


63 


581 


37 


527 


63 


570 


37 


540 


New Zeal 


66 


557 


34 


456 


68 


557 


32 


471 


Russ. Fed 


58 


494 


42 


441 


78 


489 


22 


457 


South Africa 


58 


367 


42 


353 


73 


349 


27 


366 


Sweden 


62 


583 


38 


507 


66 


586 


34 


515 


USA 


76 


476 


24 


423 


83 


491 


17 


440 



(Mullis, etal., 1998:95) 

Students were asked about their perceptions concerning how well they usually did in 
mathematics. In general, students expressed positive perceptions of their progress in 
mathematics and science. In all countries, the majority of students agreed that they were 
doing well in both subjects. Students' academic performance appears to support their 
perceptions - the average performance of those who claimed that they usually did well 
generally exceeded the performance of those who disagreed. However, whereas most 
countries had a considerable differential in mean performance scores for those who thought 
they usually did well as opposed to those that did not, in the case of South Africa this 
differential was very small for mathematics and the trend was reversed for science. In other 
words, the trend for mathematics was not that evident in South Africa, possibly suggesting that 
students' perceptions had little effect on their achievements and that, in general, they upheld 
unrealistic perceptions of their competencies. The figures for science indicate that those who 
did not think that they usually did well actually outperformed those who thought that they did 
well, and may once again point to unrealistic perceptions on the part of students. 



The gender differences in self-perceptions indicated a strong similarity between males and 
females. If any differences were evident, it was always a greater percentage of males than 
females asserting that they were doing well. 



1 1 Germany had data available for < 50% of students 
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3.4.2 Students' perceptions about liking mathematics and science subjects 

Students were asked how much they liked or disliked the science-related subjects for which 
they were enrolled. Their responses for liking mathematics are set out in Table 3.5. In almost all 
countries the majority of students reported that they liked mathematics to some degree. In 
every country, a positive relationship was observed between liking mathematics and 
achievement in mathematics. Those who disliked mathematics performed worse than those 
who expressed a liking for the subject. In general, there was no significant difference between 
males and females in their degree of liking mathematics. 

Table 3.5: Students' reports on how much they like mathematics 





Dislike a lot 


Dislike 


Like 


Like 


a lot 


Country 


% of 
students 


Mean 

mathematics 

literacy 

score 


% of 
students 


Mean 

mathematics 

literacy 

score 


% of 
students 


Mean 

mathematics 

literacy 

score 


% of 
students 


Mean 

mathematics 

literacy 

score 


Australia 


14 


455 


25 


513 


47 


538 


14 


578 


Canada 


17 


476 


22 


501 


46 


529 


15 


573 


Czech 

Republic 


19 


435 


48 


447 


28 


501 


5 


575 


Denmark 


7 


460 


14 


506 


44 


551 


34 


586 


Germany 


xx 12 


XX 


XX 


XX 


XX 


XX 


XX 


XX 


Netherland 


13 


. 


_ 


_ 


. 


. 






New Zeal 


18 


468 


29 


491 


42 


547 


1 1 


592 


Russ. Fed 


6 


442 


32 


449 


52 


480 


10 


51 1 


South Africa 


8 


334 


14 


363 


40 


367 


38 


372 


Sweden 


13 


468 


29 


521 


42 


574 


16 


625 


USA 


13 


414 


21 


446 


45 


465 


21 


509 



(Mullis, etal., 1998:99) 



Of all the sciences, students in almost all countries indicated higher levels of liking biological 
science than chemistry and physics. This was also true for South Africa and from Figure 3.2 it 
can be seen that the majority of South African students stated that they liked biology. 
Chemistry and physics seemed less appealing to students, which fits their preferences for 
further study as discussed in section 3.3.2 of this chapter. 

There were differences between the males' and females' liking of the science subjects. In the 
majority of countries female students reported liking biological science more than male students 
did. The opposite was true for physics, where more males liked physics than females. Further 
research to uncover the reasons for these findings may reveal some interesting insights. 



12 An "xx" indicates that data was available for < 50 % of students. 

13 A dash (-) indicates that data were not available. 
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Figure 3.2: South African students' reports on liking 14 science subjects 
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3.5 Educational resources in the home 

Parental education is frequently a useful indicator of the level of support for academic 
endeavour, and is often associated with student achievement. Students were asked to indicate 
the highest level of education completed by their parents. TIMSS revealed a clear positive 
correlation cross-nationally between parents' education and students' mathematics and 
science literacy. Final-year students whose parents had had more education had higher 
mathematics and science literacy scores. Along with Austria and the Netherlands, South 
Africa had the lowest proportion (1 1 %) of students whose parents had completed university. 
In the various countries this percentage ranged from 11-44%, with Canada having the 
highest proportion. Most students in South Africa reported that the highest level attained by 
either parent was to complete primary school. The bulk of the South African parents had either 
completed primary school or partly completed secondary school, and it is possible that the 
level of parental education constitutes one of the factors limiting academic performance. 



1 4 This refers to students who indicated that they either liked Mathematics or liked il a lot. 
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Table 3.6: Parental educational levels of South African students 





Father 


Mother 


Completed primary school 


24,6 


. 31,5 


Partially completed secondary school 


18,8 


22,8 


Completed secondary school 


13,3 


14,2 


Completed vocational/technical education after 
secondary school 


7,6 


5,0 


Partly completed university study 


2,6 


1,6 


Completed university study 


6,2 


4,8 


Don't know 


26,9 


20,3 



- 

That such a high proportion of students did not know the educational level of their parents was 
-^surprising, but a similar finding was established in Population 2. Perhaps it is due to the fact 
that, in general, parents with very low educational levels avoid discussing the issue. 

The number of books in the home can be an indicator of a home environment that values 
literacy and the acquisition of knowledge, and offers general academic support. The rationale 
for this question was to gain information about the relative importance of academic pursuits in 
the students' home environments, rather than to determine the actual number of books in the 
home. Close to 60 % of students in South Africa had less than 26 books in their home. This 
means that the bulk of sampled students in South Africa had only a few books in their home. 
Compared to other countries South Africa had the highest proportion (almost 60 %) of 
students reporting that they had less than 26 books in the home. Most countries had between 
5 and 23 % of students who reported having this amount of books at home. At the other end 
of the scale more than 40 % of the students in countries such as Australia, Denmark, Norway 
and Sweden reported more than 200 books in their homes. 

Asked whether they had calculators in their home almost 47 % of South African students 
reported that they had, whilst about 54 % did not. In terms of how often calculators are used 
at school, home or anywhere else, the majority of students in every country indicated that they 
use it weekly or daily. Additionally, students in the majority of countries said that they used 
calculators for some purpose on a daily basis. 
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Figure 3.3: Students's reports on the frequency of calculator use at school, home or 
anywhere else 
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In South Africa 92 % of students indicated that they did not have a computer at home. 
Concerning the frequency of computer use at school, home or anywhere else, 81 % of South 
African students said that they rarely or never used a computer. In six other countries including 
Cyprus, the Czech Republic, Hungary, Italy, Lithuania and Norway, more than 50 % of 
students reported that they rarely or never used a computer, although South Africa showed the 
highest proportion by far. 15 A number of countries such as Australia, Canada, the 
Netherlands, New Zealand, Switzerland and the United States had the highest proportion 
of students indicating that they used computers daily. This proportion ranged from 28-31 %. 



15 The next highest proportion was 69 % (Lithuania). 
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3.6 Use of out-of-school time 

3.6.1 Homework 

Young people spend a large proportion of their time outside school. Some of this time is spent 
furthering academic development. Concerning homework, students in most countries reported 
spending, on average, between two and three hours a day on homework. South Africa stood 
out in that its students reported spending about five hours a day on homework (see Figure 3.4). 
This was the highest average time devoted to homework, compared to all the participating 
countries. 



Figure 3.4: Average 16 hours spent on doing homework per day 
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The relationship between time spent doing homework and students' average mathematics and 
science literacy performance was inconsistent and trends were difficult to identify. In a few 
countries a linear relationship was evident but, as in the South African case, more time devoted 
to homework did not result in higher scores. This can be seen as one of the contradictory 
features emerging from the South African data and raises questions concerning students' ability 
to estimate as well as other possible explanations. More frequently, the highest achievement 
was associated with a moderate amount of homework per day. This optimal level appeared to 
be between one and three hours. 



■ i — 




16 The value was calculated based on the following: No time = 0, less than 1 hour = 0.5, 1-2 hours = 1 .5, 3-5 
hours = 4, more than 5 hours = 7. 
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The amount of time students reported spending on studying or doing homework each day in 
mathematics and science also revealed some interesting peculiarities. An average of 0.5-1 
hour of homework in both subject areas was reported in nearly all countries. South African 
students, on the other hand, spent an average of 1.7 hours per day on mathematics 
homework and 1 .5 hours on science homework. In each case this was the highest proportion 
of time spent on homework compared to all the other countries. Despite spending a 
considerable amount of time on homework in general, and on specific subjects, this additional 
out-of-school work does not appear to enhance performance levels for South African students. 
Perhaps the low performance in the literacy test says something about students' general and 
applied knowledge in mathematics and science. It is likely that South African students are 
accustomed to syllabus-related work and more particularly, specific questions which are 
commonly practised in class and appear in examinations. The TIMSS literacy test comprised 
applied word problems and not the conventional question types that students are ordinarily 
faced with in their school settings. Thus, the low performance of South African students can in 
part be explained by their generally low literacy levels in mathematics and science, as well as 
their competencies in handling tests which contain largely word problems. 



Figure 3.5: Average hours spent on mathematics and science homework per day 
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3.6.2 Other activities 

Students were also asked about the various ways in which they spent their out-of-school time. 
The activity occupying more of South African students' time than all other activities was doing 
jobs at home (see Table 3.5). Compared to students in other countries, South African students 
spent the most time on this task. The activity that absorbed the least amount of South African 
students' time was playing computer games. Eighty-four percent indicated that they spent no 
time on this activity. Another activity which was not a significant part of South African students' 
lives was working at a paid job. Seventy-five percent of the students indicated that they did 
not engage in this activity or spent little time on this activity after school. As alluded to earlier, it 
appears that South African students spend far more time on homework than on other out-of- 
school activities. 

Table 3.7: South African students' reports on how they spend their leisure time 



Activity 


Time 

(Average hours per 
day) 


Watching television or videos 


1,2 


Playing computer games 


0,2 


Spending time with friends outside of school 


U 


Doing jobs at home 


2,2 


Working at a paid job 


0,9 


Playing sports 


1,3 


Reading a book for enjoyment 


1,3 



3.7 Conclusion 

This chapter examined the background features that surround students' learning and home 
environments. From the TIMSS literacy study definite positive relationships were identified e.g. 
academic performance is supported by student perceptions and liking mathematics is 
positively related to higher achievement. In a number of ways South Africa stands out - their 
students were almost the oldest and they spent the most time on household chores, students' 
homes boasted far fewer books, and students' parents had close to the lowest educational 
levels. These unique features (among others) are likely to limit academic performance 
considerably. In addition, a number of surprising or contradictory elements have emerged from 
the South African analysis. These include the excessive amount of time spent on homework 
without any return on performance level, and students' perceptions of their good performance 
in mathematics and science and the disjuncture with their performance in the literacy test. The 
identification of the dominant factors influencing the low literacy levels in mathematics and 
science, and how these levels can be improved, are important components of the detailed 
analysis of TIMSS that will be undertaken in later research reports. 
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Chapter 4 

Overall achievement in 
mathematics and science 
literacy 

Key points 

♦ The overall scores were significantly lower than those 
of students in other countries, 

♦ South Africa was the only country with no significant 
difference between the performance of boys and 
girls. 

♦ Free State, Gauteng and KwaZulu-Natal performed 
consistently better than the other 6 provinces. 

♦ South African students with English and Afrikaans as 
a home language performed significantly better than 
those students with other home languages. 
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Chapter 4 

Overall achievement in mathematics 
and science literacy 



4.1 National mathematics and science literacy achievement 



In this chapter, the results of the performance of the South African Grade 12 students are 
presented and compared to with those of students in ten other countries participating in TIMSS. 
In total, 22 countries participated in TIMSS Population 3. The ten countries were selected from 
the top-scoring, middle-scoring and bottom-scoring countries on the achievement results scale. 
These results reflect the students performance in the mathematics and science literacy test. This 
test was designed to measure the mathematics and science literacy of all final-year students, 
regardless of their school curriculum. In other words, the test was applied to both students 
taking a science subject 17 and those who did not. In the case of the South African students 
presented in this report, 1 769 students took a science subject while 482 students did not. The 
study was designed so that school leavers could be assessed as a whole in terms of their 
readiness to apply their mathematics and science knowledge in real life situations. 

Figure 4. 1 presents the students' average score achieved in this test. South Africa is compared 
here to 10 other countries. As highlighted in the International report, South Africa obtained a 
significantly lower average score (352 points) than the other participating countries for which 
the international average score was 500 on a scale of 800 points. 



; Figure 4. 1 : South African students' average mathematics and sciences literacy score in 
i comparison with other countries 
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This included mathematics, physical science, biology and geography on either standard grade or higher 
grade. 
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4.2 The comparative performance of the top 25 % of South 
African students 

Figure 4.2 below represents the average achievement of the top 25 % of the students in the 
entire school-leaving age cohort for South Africa and ten other countries. Here South African 
students' performance was also lower than that of the other participating countries, achieving 
328 points compared to the International average of 520 points. 



Figure 4.2: 



Comparative performance of the top 25% of South African students 
(average scores) 
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4.3 The comparative performance of South African students by 
gender 

Figure 4.3 shows the differences in the students' achievement by gender. Internationally, the 
male students performed significantly better than the female students, achieving 519 
compared to 483 pints respectively. This was not the case in South Africa where there was 
no significant difference (males 366, females 341 ). This was also the case in the TIMSS results 
for South African Grade 7 and Grade 8 students. 
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Figure 4.3: Comparative performance of South African students by gender (average 
scores) 

700 



600 




male \ ; female 



4.4 Students' performance compared per province 

The results for the provinces were significantly different in some cases. As can be seen from 
Table 4.1 below, the provinces that performed above the South African national average 
score were Free State, Gauteng, KwaZulu-Natal, Northern Cape and Western Cape. The 
Northern Cape and Western Cape scored well above the national average of 352 points 
(and the Western Cape students' average was well above the international average of 500 
points). 18 

The remaining provinces scored below the national average (i.e. Mpumalanga, Northern 
Province, North West and Eastern Cape). The results of this group, with the possible 
exception of the North West Province, are not surprising considering the current conditions in 
education in these provinces. 

Despite some of the low scores that were achieved, there were also exceptionally high scores 
obtained. It is encouraging to note that the maximum scores of students in the Free State (593), 
Gauteng (642), KwaZulu-Natal (700), Northern Cape (502) and Western Cape (761 ) were 
above the international average (500). In the case of students in KwaZulu-Natal and the 
Western Cape, exceptionally high scores were achieved. Further research will be conducted 
to investigate the reasons for these very good results as well as the poor results achieved by 
students in the rest of South Africa. 



18 Due to the small size of the samples in both of these provinces, these results cannot be considered 
representative. 
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Table 4.1 : Students' achievement per province 19 



Province 


Average score 
obtained 


Minimum 

score 


Maximum 

score 


Eastern Cape 


315 


195 


422 


Free State 


409 


229 


593 


Gauteng 


370 


236 


642 


Kwazulu-Natal 


395 


206 


700 


Mpumlanga 


342 


194 


583 


Northern Province 


305 


165,5 


487 


North-West 


321 


223 


434 



4.5 Students with a science subject compared to students with no 
science subject 

As the sample of students who were tested also included students who did not take any 
science subject at all, it was possible to compare those students' literacy levels with students 
who had taken at least one science subject for matriculation (Grade 1 2). Students were asked 
to indicate which of the following subject choices they were taking: 

♦ Mathematics (higher grade) and physical science (higher grade) 

♦ Mathematics (standard grade) and physical science (higher grade) 

♦ Mathematics (higher grade) and physical science (standard grade) 

♦ Mathematics (higher grade) and no physical science 

♦ Mathematics (standard grade) and physical science (standard grade) 

♦ No mathematics and no physical science 



19 The figures for the remaining provinces are shown below in the same format as Table 4.1 



Northern Cope 


458 


409 


502 


Western Cope 


618 


468 


761 
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The results of the performance of these different groups are presented below: 



Table 4.2: South African students' performance per subject option 



Subject option 


Average score 


minimum 

score 


maximum 

score 


Mathematics (higher grade) and Physical 
science (higher grade) 


397 


215 


761 


Mathematics (standard grade) and 
Physical science (higher grade) 


353 


195 


643 


Mathematics (higher grade) and Physical 
science (standard grade) 


334 


203 


680 


Mathematics (higher grade) and no 
Physical science 


332 


194 


640 


Mathematics (standard grade) and 
Physical science (standard grade) 


380 


200 


623 


No mathematics and no Physical science 


320 


223 


51 1 


Overall National Score 


352 



As expected, the average scores indicated that the students taking the option of higher grade 
mathematics and higher grade science achieved the highest results; and the students not 
taking either mathematics or science achieved the poorest results. However, on closer 
analysis, a number of oddities emerge. The average score of those doing no mathematics and 
science compared to that of students taking mathematics with physical science (standard 
grade) or without physical science, shows a very small differential (12-14 points). This raises 
questions about the value of doing either formal mathematics or physical science in relation to 
impacting on literacy in these subjects. Interestingly, the lowest scores were attained by 
students who were in fact taking both mathematics and physical science. Another surprising 
observation was the fact that students taking mathematics and science on standard grade 
performed better than students taking standard grade mathematics and higher grade science. 
All in all, the fact that little value is added to students' general scientific literacy by enrolling for 
an additional mathematics or science subject is testimony to the poverty of our current system. 
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4.6 Students 7 performance related to their use of the language of 
instruction 

Students were tested in their language of instruction, which in 1995 was either English or 
Afrikaans. Students were asked to indicate the degree to which they spoke the language of 
instruction at home. From these responses, Table 4.3 was compiled, indicating whether or not 
students spoke the language of instruction at home always, sometimes or never. It would 
appear from the average scores that the students who did speak the language of instruction as 
their home language performed significantly better overall than those students who either did 
not speak the language of instruction at home or sometimes did. As this difference can be 
attributed to a number of reasons (e.g. students speaking English/Afrikaans at home may also 
have come from better socio-economic backgrounds), there is no simplistic way to explain this. 
Despite this result, there were students in the groups who never spoke the language of 
instruction at home, and some who sometimes did, who achieved exceptionally high results 
that were well above the international average. The whole question of language and 
performance will be analysed in detail through further research. 

Table 4.3: Students' performance compared to their use of the language of instruction 



Students use of 
language of 
instruction at home 


Number of students 


Average score obtained 


always 


502 


442 


sometimes 


1 913 


317 


never 


251 


318,5 



4.7 Conclusion 

Although the South African students did not perform well overall, there were individual 
students who did excel and achieved very high scores. It also encouraging to note that there 
was no significant difference between the boys and girls in terms of their mathematics and 
science literacy scores. It was also interesting to see the difference in the students' scores 
between the provinces and policy-makers will find these results of particular interest. 
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Chapter 5 



Achievement in mathematics 
literacy and science literacy 

Key points 

♦ South African students performed significantly worse 
overall in the mathematics and science literacy test. 

♦ South African students performed marginally better 
in mathematics than science. 

♦ There was no significant difference in the 
performance of boys and girls in either 
mathematics or science. 

♦ Students in the KwaZulu-Natal and the Free State 
provinces performed above the national average for 
mathematics and science. 
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Chapter 5 

Achievement in mathematics literacy and science literacy 

This chapter focuses on students' performance in the areas of mathematics literacy and 
science literacy, separately, and discusses the results by gender, province and language. 



5.1 Performance of South African students in mathematics 
literacy 

The mathematics literacy test items included number sense (including fractions, percentages 
and proportionality), algebraic sense, measurement and estimation, data representation and 
analysis. Reasoning and social utility were emphasised in several items. Mathematics 
questions were intended to be relevant to real life situations. 

The figure below presents the results of the South African students compared to 10 other 
countries. A number of countries, including Denmark, performed significantly better in 
mathematics than in science. The South African students performed marginally better in the 
mathematics literacy items than they did in the science literacy items, achieving a score of 356 
points compared to the international average of 500 poinfs. 



Figure 5.1 : Comparative performance of South African students in mathematics 
literacy 
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5.2 Performance of South African students in science literacy 

The science items included the areas of earth science, life science and physical science. The 
items also included a reasoning and social utility component. The use of calculators was 
permitted for this test. The Figure 5.2 below presents the results of the South Africa students 
compared to 10 other countries. Several countries, including Sweden, the Russian Federation, 
the Czech Republic, Canada and the USA, performed significantly better in the science 
literacy items than the mathematics. South African students' performance was, again, lower 
than that of all the other participating countries, achieving a score of 349 points compared to 
the international average of 500 points. 



Figure 5.2: Comparative performance of South African students in science literacy 
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5.3 South African students' comparative performance in 
mathematics and science literacy by gender 

There were significant differences between the performances of male and female students 
internationally in both mathematics and science. However, in science there was a greater 
difference than in mathematics. This was true for all countries except South Africa, where there 
was no significant difference in either mathematics or science. 
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Figure 5.3: South African students' comparative performance in 
by gender 
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Figure 5.4: South African student's comparative performance in 
by gender 
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5.4 The performance of South African students, by province, in 
mathematics literacy and science literacy 

The performance of the South African students varied considerably across the different 
provinces. In Table 5.1 below the students' scores by province and subject are presented. In 
the mathematics literacy items, the students in the Eastern Cape, Mpumalanga, Northern and 
North West provinces performed below the national average. The students in KwaZulu-Natal 
and the Free State province performed well above the national average. Although the scores 
for the Northern Cape and especially the Western Cape appeared to be well above the 
national average, these scores (as mentioned previously) cannot be seen to be representative 
due to the very small size of both these samples. 

In the science literacy items, as with the mathematics literacy items, students from the Eastern 
Cape, Mpumalanga, Northern Province and the North West Province performed below the 
national average. Once again, the students from KwaZulu-Natal and the Free State 
performed well above the national average. 



Table 5.1 : Comparative provincial scores per subject 



Province 20 


Maths and science 
literacy average 
score 


Mathematics 
literacy average 
score 


Science literacy 
average score 


Eastern Cape 


316 


322 


310,5 


Free State 


410 


39Z 


423 


Gauteng 


3Z8 


328 


3Z4 


Kwazulu-Natal 


404 


403 


405 


Mpumalanga 


346 


349,5 


342 


Northern Province 


308 


316 


300 


North West . 


319 


325 


312 


South Africa 


352 


356 


349 



20 The scores for the Northern Cape and Western Cape scores cannot be considered representative and 
therefore are not included in Table 5.1 and are presented below: 



Northern Cope 


457.5 


439 


476 


Western Cape 


618 


612 


624 
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5.5 South African students' performance in mathematics literacy 
and science literacy, by language of instruction used 

The students were asked to state the extent to which they spoke the language of the test 
(which was the language of instruction at their school) at home, (i.e. always, sometimes or 
never). The results are presented in Table 5.2, by province and by language of instruction. 

There are several surprising results from this analysis. First, there seemed to be a difference 
between the students using the language of the test in four out of the nine provinces, namely 
Free State, Gauteng, KwaZulu-Natal and Mpumlanga. In three of these provinces, namely 
Free State, Gauteng and KwaZulu-Natal, the students' overall scores, science literacy scores 
and mathematics scores were better than the other provinces. Based on the average scores, 
the students from the Western Cape and the Northern Cape, although their scores were not 
representative of the province, had scores that were well above the national average and did 
not seem to follow the same pattern- as the Free State, Gauteng, KwaZulu-Natal and 
Mpumlanga scores. In the Western Cape and Northern Cape the findings suggest that 
students did not appear to be disadvantaged by the language of the test. For instance, the 
students in the Northern Cape who sometimes spoke the language of the test had better scores 
in both the mathematics literacy and science literacy than the students who always spoke the 
language at home. In the Western Cape, students who never spoke the language at home 
achieved higher scores than those who always spoke the language at home. These 
unexpected findings will need investigation. 

In the Northern Province, the Eastern Cape and North West Province, the situation appears to 
be more complex. There was no relation between the students use of language at home and 
their performance in either mathematics or science. In fact, the scores were poor across all 
three provinces and across the three groups of language use at home. This suggests that there 
appears to have been serious inadequacy in the learning and teaching of science and 
mathematics, which is pervasive throughout these students' school careers in these provinces. 
The low levels of literacy in mathematics and science should be alarming to the policy makers 
in these provinces, especially in the Eastern Cape and Northern Province which also have the 
highest unemployment figures in the country. 

The relationship between language and performance in mathematics and science is an issue 
that will be investigated as a project on its own as it is perceived to be not only a complex 
question, but also a critical issue. 
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Table 5.2 The performance of students by their use of the language of the test 



Province 21 


Use of language 
of instruction 


Mathematics 
average literacy 
score 


Science average 
literacy score 


Eastern Cape 


always 


315 


319 




sometimes 


323 


310 




never 


315 


303 


Free State 


always 


436 


482 




sometimes 


324 


320 




never 


334,5 


320,5 


Gauteng 


always 


463 


471 




sometimes 


342 


327 




never 


361 


333 


Kwazulu-Natal 


always 


451 


464 




sometimes 


331 


317 




never 


319 


306 


Mpumalanga 


always 


441 


466 




sometimes 


317 


299 




never 


323 


302 


Northern 


always 


316 


307 




sometimes 


314 


292 




never 


321 


308 


North West 


always 


337 


336 




sometimes 


325 


309 




never 


337 


312 



21 The Northern Cape and Western Cane Province were not included in Table 5.2 as these scores were not 
representative due to the small size of the sample. These provinces'scores are presented below: 



Northern Cape 


always 


434.5 


473 




sometimes 


447 


484 


Western Cape 


always 


622 


628 




sometimes 


541 


592 




never 


617 


636 
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5.6 Conclusion 

This chapter provided a breakdown of the mathematics literacy scores and the science literacy 
scores, and examined the factors involved. Much more research will need to be conducted to 
investigate issues raised in this chapter, particularly with reference to the provincial analysis 
and the reasons why South African students performed better in mathematics literacy than in 
science literacy. 
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Chapter 6 



Performance on 
mathematics and science 
literacy items 

Key points 

♦ On all the items presented the performance of South 
African students was the lowest. 

♦ South African students appear to have difficulty with 
graphic interpretation. 

♦ In general South African students experienced great 
difficulty in articulating explanations for the free- 
response items. 
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Chapter 6 



Performance on mathematics and science literacy items 



6. 1 Preamble 

Apart from the overall achievement results and an understanding of how students performed in 
the broad content areas, it is also useful to examine students' performance on specific items. 
This chapter explores performance in four example items within both the mathematics and 
science literacy content areas. Although the selection of these items attempted to include as 
broad a range of topics as possible, this was limited due to the few items examined. 

The literacy test was designed to test students' general mathematical and scientific 
knowledge. The items were selected on the basis that they reflected questions that relate to 
real life situations. 

Three types of questions were included in the literacy test - multiple-choice, short answer and 
extended response items. The two latter types of questions expected students to generate and 
articulate their own responses. Scoring of free-response items rested on whether the response 
was completely or only partially correct. Two to three marks were awarded for a fully correct 
response. For the multiple- choice questions most questions were worth one point. The table 
below indicates the weighting for the different types of questions within the literacy test. 



Table 6.1 : Distribution of mathematics and science literacy items by reporting 
category 



Content category 


% 

of items 


Number 
of items 


Number 
of multiple 
choice 
items 


Number 
of short 
answer 
items 


Number 
of extended 
response 
items 


Number 
of score 
points 


Mathematics literacy 


58 % 


44 


34 


8 


2 


53 


Science literacy 


42 % 


32 


18 


9 


5 


43 


TOTAL 


100 % 


76 


52 


17 


7 


96 



(Mullis, 1 998:B-9) 



6.2 Example items in mathematics literacy 

In general the items within the mathematics literacy content area were designed to define 
adequately the content area and restrict the items to the content areas most closely related to 
mathematical literacy. In this chapter items relating to number sense, data representation and 
estimation are included to illustrate the types of questions appearing in the literacy tests and 
the performance attained on these items. Some items reflect the reasoning skills expected of 
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students. The items emphasise the types of understanding and skills needed for effective 
participation in today's information-rich society. Figure 6. 1 illustrates the comparison between 
the South African scores and the international averages for the mathematics example items. 



Figure 6.1 : Comparison between SA scores 22 and the international averages for the 
mathematics example items 
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Example Item 1 : 

If there are 300 calories in 100 grams of a certain food, how many 
calories are there in a 30 gram portion of that food? 



A. 


90 


B. 


100 


C. 


900 


D. 


1000 



This question is a straightforward proportionality problem, which receives significant attention 
within the South African curriculum, at least by Grade 9. Most countries fared reasonably well 
on this item. The average percentage of correct answers across the participating countries 
was 71 %, with scores ranging from 45-84 %. South Africa performed well below the 
international average at 45 %. Table 6.2 gives a comparison of South African students' 
performance on this item with some other countries. 



22 Only fully correct responses are indicated on the graph. 
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Table 6.2: Mathematics literacy: percent correct for example item 1 



Country 


Percent 

correct 


Australia 


79 


Canada 


73 


Czech Republic 


61 


Denmark 


75 


Germany 


74 


Netherlands 


84 


New Zealand 


75 


Russian Federation 


71 


Sweden 


74 


USA 


68 


South Africa 


45 


International average percent correct 


71 



Example Item 2: 

Kelly went for a drive in her car. During the drive, a cat ran in front of the car. Kelly slammed on 
the brakes and missed the cat. 



Slightly shaken, Kelly decided to return home by a shorter route. The graph below is a record of 
the car's speed during the drive. 

Kelly's drive 




Time 



(a) What was the maximum speed of the car during the drive? 

(b) What time was it when Kelly slammed on the brakes to avoid the cat? 
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In this item students were expected to interpret a graph and then to provide responses to the 
questions. In the first part they needed to be able to read a line graph and use the axes 
information to determine the maximum speed. The international average of correct responses 
was 74 %, whereas in South Africa 60 % of students responded correctly. These scores 
indicate that students found this part of the question relatively easy. The second part of the 
problem which involved further interpretation of the information in the graph proved to be more 
difficult for students. Whereas the international average was 59 %, only 1 9 % of South African 
students got the answer correct. This could imply that South African students have difficulty in 
understanding graphs, particularly the seemingly more complex sections where the graph line 
descends. The range of correct responses internationally was 19-83 %. 

Table 6.3: Mathematics literacy: percent correct for example Item 2 



Country 


Percent correct 
(part A) 


Percent correct 
(part B) 


Australia 


88 


68 


Canada 


80 


67 


Czech Republic 


66 


47 


Denmark 


78 


67 


Germany 


74 


62 


Netherlands 


91 


83 


New Zealand 


91 


74 


Russian Federation 


62 


46 


Sweden 


85 


69 


USA 


85 


67 


South Africa 


60 


.19 


International average percent correct 


74 


59 



Example Item 3: 

Brighto soap powder is packed in cube-shaped cartons. A carton measures 
1 0 cm on each side. The company decides to increase the length of each edge 
of the carton by 1 0 per cent. How much does the volume increase? 

A. 10 cm 3 

B. 21 cm 3 

C. 100 cm 3 

D. 331 cm 3 
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This problem involves percentages and volume, and requires a number of steps to solve. From 
the results it can be assumed that the students found this a difficult problem. Fewer than half of 
the students selected the correct solution in each of the participating countries. In South Africa 
only 6 % of the students selected the correct answer and the international average at 31 % is 
also very low. Difficulties with this item could be linked to the complexities of multi-stepped 
approaches to solving problems, the calculation of percentages or the concept of volume. 

Table 6.4: Mathematics literacy: percent correct for example Item 3 



Country 


Percent 

correct 


Australia 


30 


Canada 


29 


Czech Republic 


21 


Denmark 


41 


Germany 


25 


Netherlands 


50 


New Zealand 


36 


Russian Federation 


30 


Sweden 


41 


USA 


17 


South Africa 


6 


International average percent correct 


31 



Example item 4: 

A TV reporter showed this graph and said: "There's been a huge increase in the number of 
robberies this year". 
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This item is an open-ended question asking students to comment on a reporter's interpretation 
of a graph. On average, 19 % of the students across countries received full credit for their 
responses. Another 26 % received partial credit 23 for their responses. Of the items presented 
in this chapter the performance scores for this item are the lowest. The international average 
for fully correct answers was 1 9 % and the average for South Africa was 3 %. Clearly students 
in general found this type of open-ended, free-response item problematic, and had difficulty in 
articulating responses. 

Table 6.5: Mathematics literacy: percent correct for example item 4 



Country 


Percent partially 
correct 


Percent 
fully correct 


Australia 


39 


26 


Canada 


35 


23 


Czech Republic 


26 


6 


Denmark 


25 


26 


Germany 


26 


20 


Netherlands 


27 


30 


New Zealand 


38 


33 


Russian Federation 


13 


7 


Sweden 


29 


37 


USA 


41 


14 


South Africa 


12 


3 


Internotionol overage percent correct 


26 


19 



V 



6.3 Performance on science literacy 

The science literacy content areas covered earth science, physical science (energy and other 
physical science) and life science (human biology and other life science). The emphasis in 
testing in this section of the literacy test embraced how students could use their knowledge in 
response to science-related problems. Figure 6. 1 illustrates the South African scores for the 4 
example items compared to the international averages. 



23 Free-response items were coded according to whether they were entirely correct, partially correct or 
incorrect. 
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Figure 6.2: Comparison between SA scores 24 and the international averages for the 
science example items 
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Example Item 1 : 

It is often claimed that “cooked vegetables are not as nutritious as the same 
kinds of vegetables uncooked". What could be done to find out if this 
statement is true? 

A. Compare the weight of the vegetables before and after they are cooked. 

B. Compare the colour of the cooked and uncooked vegetables. 

C. Test the acidity of the water in which the vegetables are cooked. 

D. Compare the vitamin content of the cooked and uncooked vegetables. 



This item required an understanding of the links between vitamin content and nutrition. It was 
one of the easier items in the science section. The international average of correct responses 
was 87 % and the proportion of students selecting the correct answer in South Africa was 
55 %. The South African score was significantly lower than those of all the participating 
countries, of which the next lowest score was 81 %. 



24 Only fully correct scores are indicated on the graph. 
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Table 6.6: Science literacy: percent correct for example Item 1 



Country 


Percent 

correct 


Australia 


89 


Canada 


91 


Czech Republic 


92 


Denmark 


93 


Germany 


87 


Netherlands 


' 89 


New Zealand 


86 


Russian Federation 


88 


Sweden 


90 


USA 


81 


South Africa 


55 


International average percent correct 


87 



Example item 2: i 

CFC's (chlorofluorocarbons) revolutionised personal and industrial life for 
30 years. They were in refrigerators and the propellants in aerosols, pressure ! 
packs and fire extinguishers. There are now very strong international moves to i 
stop the use of these substances because j 

A. they are chemically inert j 

B. they contribute to the greenhouse effect j 

C. they are poisonous to humans 

D. they destroy the ozone layer. j 

■ i 

i 



This item focused on the issue of chlorofluorocarbons polluting the atmosphere. Again this 
question was answered correctly by a large proportion of students in all the participating 
countries with the international average percent correct at 77 %. However, a low proportion 
of South African students (39 %) obtained the correct solution. 
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Table 6.7: Science literacy: percent correct for example Item 2 



Country 


Percent 

correct 


Australia 


69 


Canada 


84 


Czech Republic 


92 


Denmark 


83 


Germany 


66 


Netherlands 


89 


New Zealand 


79 


Russian Federation 


66 


Sweden 


93 


USA 


77 


South Africa 


39 


International average percent correct 


77 



f 



Example Item 3: 

Some high heeled shoes are claimed to damage floors. The base 
diameter of these very high heels is about 0.5 cm and of ordinary 
heels about 3 cm. 



Briefly explain why the very high heels may cause damage to floors. 



As this was a free-response item students received credit for partially correct answers. The 
international average for partially correct answers was 20 % and for fully correct answers 
41 %. South Africa had 9 % providing partially correct answers and 1 0 % offering fully correct 
answers. 
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Table 6.8: Science literacy: percent correct for example Item 3 



Country 


Percent partially 
correct 


Percent 
fully correct 


Australia 


17 


53 


Canada 


18 


51 


Czech Republic 


22 


28 


Denmark 


25 


39 


Germany 


13 


52 


Netherlands 


23 


55 


New Zealand 


23 


45 


Russian Federation 


22 


31 


Sweden 


24 


47 


USA 


18 


24 


South Africa 


9 


10 


International average percent correct 


20 


41 



Example Item 4: 

Nuclear energy can be generated by fission or fusion. Fusion is not 
currently being used in reactors as an energy source. Why is this so? 

A. The scientific principles on which fusion is based are not yet known. 

B. The technological processes for using fusion safely are not yet developed. 

C. The necessary raw materials are not readily available. 

D. Waste products from the fusion process are too dangerous. 



The achievement scores on this item ranged from 26-51 %. As such the scores on this item 
were not particularly high. The international average percent correct was 40 % and 26 % of 
South African students selected the correct solution. 
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Table 6.9: Science literacy: percent correct for example item 4 



Country 


Percent 

correct 


Australia 


42 


Canada 


40 


Czech Republic 


38 


Denmark 


51 


Germany 


44 


Netherlands 


41 


New Zealand 


37 


Russian Federation 


50 


Sweden 


54 


USA 


41 


South Africa 


26 


International average percent correct 


40 



6.4 Conclusion 

On all the items discussed, the performance of South African students was the lowest among 
the participating countries. South African students do not appear to perform very well in the 
area of graphic interpretation. Their performance in multiple-choice questions seems to be a 
function of the level of difficulty or complexity of the item. In general, South African students 
also experienced great difficulty in articulating explanations for the free-response items. More 
in-depth analysis concerning how students do on multiple-choice questions as opposed to 
free-response items, and their responses to the test items, is likely to uncover some of the 
misconceptions held by the students, and areas of difficulty they experience. 
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Conclusion 



International comparative studies provide a broad understanding of different educational 
systems. Of immense benefit derived from these kinds of studies is the insight obtained 
concerning the curricula followed in various countries, the contexts supporting the learning of 
these curricula and the performance of students as a result of being exposed to particular 
curricula. In the past, the findings of international comparative studies and their subsequent 
policy recommendations have catalysed various countries to introduce educational reforms. In 
South Africa, this study starts to address the dearth of national data concerning mathematics 
and science education. The results of TIMSS thus constitute useful baseline information which 
can inform educational planning for the next few years. 

The overall results for final-year students taking the mathematics and science literacy test are 
somewhat bleak. The fact that South African students performed so poorly in the test, as well 
as on individual items, suggests that their general mathematics and science understanding and 
skills are limited, falling short of literacy levels necessary for effective functioning in society. 
TIMSS provides policy makers and practitioners with the status quo in mathematics and science 
competencies. Definite benefits are to be gained from monitoring the progress of these 
competencies within a longitudinal study (every 3 years). It would be particularly useful to 
establish the impact of "Curriculum 2005" in terms of improving the types of skills that were 
tested in the TIMSS literacy test. Such insight can serve to aid the government in the 
assessment of its current policy initiatives. 

From the preliminary TIMSS analysis, a number of issues have already emerged which beg 
either secondary analysis or further research. The issue of taking word-problem-oriented tests 
in a second or third language should be examined, to establish the extent to which this impacts 
on performance levels. The question of why a considerable amount of time spent on 
homework can add very little value to performance levels in mathematics and science literacy 
needs exploration. Students taking mathematics and science did not perform a lot better than 
those not taking these subjects. Does this imply that formal schooling in these subjects does not 
prepare young people with the numerical and scientific literacy needed for the world of work 
(or even tertiary studies)? More in-depth analysis of how students do on multiple-choice items 
as opposed to free-response items is likely to provide insight in terms of the misconceptions 
held, and areas of difficulty faced by the students. This can serve to inform both curriculum and 
textbook developers, as well as educational practitioners. Finally, the dominant factors 
impacting on performance levels in mathematics and science literacy in South Africa could be 
delineated. There is a host of information linked to this that can be uncovered at provincial 
level. 

If South Africa is to succeed as a country in a rapidly changing competitive world in which 
science, engineering and technology are becoming increasingly important, a premium must be 
placed on science and technology education. The development of human resources in these 
fields is of utmost importance. This study is useful as a frame of reference for monitoring 
purposes when programmes, reforms and other educational initiatives linked to mathematics 
and science education are implemented. The TIMSS study in South Africa provides the 
necessary information to contribute to finding solutions to the educational challenges that 
plague our country at present. 
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